{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Split Learning—Bank Marketing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*The following codes are demos only. It's **NOT for production** due to system security concerns, please **DO NOT** use it directly in production.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this tutorial, we will use the bank's marketing model as an example to show how to accomplish split learning in vertical scenarios under the `SecretFlow` framework.\n",
    "`SecretFlow` provides a user-friendly Api that makes it easy to apply your Keras model or PyTorch model to split learning scenarios to complete joint modeling tasks for vertical scenarios.\n",
    "\n",
    "In this tutorial we will show you how to turn your existing 'Keras' model into a split learning model under `Secretflow` to complete federated multi-party modeling tasks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is Split Learning？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The core idea of split learning is split the network structure. Each device (silo) retains only a part of the network structure, and the sub-network structure of all devices is combined together to form a complete network model. \n",
    "In the training process, different devices (silo) only perform forward or reverse calculation on the local network structure, and transfer the calculation results to the next device. Multiple devices complete the training through joint model until convergence.\n",
    " <img alt=\"split_learning_tutorial.png\" src=\"resource/split_learning_tutorial.png\" width=\"600\">  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Alice`：have `data_alice`，`model_base_alice`  \n",
    "`Bob`: have `data_bob`，`model_base_bob`，`model_fuse`  \n",
    "\n",
    "1. `Alice` uses its data to get 'hidden0' through 'model_base_Alice' and send it to Bob . \n",
    "2. `Bob` gets `hidden1` with its data through `model_base_bob`.\n",
    "3. `hidden0` and `hidden1` are input to the 'Agg Layer' for aggregation, and the aggregated 'hidden_merge' is the output.\n",
    "4. `Bob` input `hidden_merge` to `model_fuse`, get the gradient with `label` and send it back.\n",
    "5. The gradient is split into two parts' g0 ', 'g1' through 'AggLayer', which are sent to 'Alice' and 'Bob' respectively.\n",
    "6. Then `Alice` and `Bob` update their local base net with `g0` or `g1`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Marketing is the banking industry in the ever-changing market environment, to meet the needs of customers, to achieve business objectives of the overall operation and sales activities. In the current environment of big data, data analysis provides a more effective analysis means for the banking industry. Customer demand analysis, understanding of target market trends and more macro market strategies can provide the basis and direction.  \n",
    "  \n",
    "The data from [kaggle](https://www.kaggle.com/janiobachmann/bank-marketing-dataset)is a set of classic marketing data bank, is a Portuguese bank agency telephone direct marketing activities, The target variable is whether the customer subscribes to deposit product."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data\n",
    "\n",
    "1. The total sample size was 11162, including 8929 training set and 2233 test set\n",
    "2. Feature dim is 16, target is binary classification\n",
    "3. We have cut the data in advance. Alice holds the 4-dimensional basic attribute features, Bob holds the 12-dimensional bank transaction features, and only Alice holds the corresponding label"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by looking at what our bank's marketing data look like?  \n",
    "\n",
    "The original data is divided into Bank_Alice and Bank_Bob, which stores in Alice and Bob respectively. Here, CSV is the original data that has only been separated without pre-processing, we will use `secretflow preprocess` for FedData preprocess"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-06-29 11:39:09.253448: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib:/opt/rh/devtoolset-10/root/usr/lib64/dyninst:/opt/rh/devtoolset-10/root/usr/lib/dyninst:/opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib\n",
      "E0629 11:39:12.246428698 2229130 fork_posix.cc:70]           Fork support is only compatible with the epoll1 and poll polling strategies\n",
      "E0629 11:39:12.271053599 2229130 fork_posix.cc:70]           Fork support is only compatible with the epoll1 and poll polling strategies\n",
      "E0629 11:39:12.286032004 2229130 fork_posix.cc:70]           Fork support is only compatible with the epoll1 and poll polling strategies\n"
     ]
    }
   ],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "import secretflow as sf\n",
    "\n",
    "sf.init(['alice', 'bob'], num_cpus=8, log_to_driver=True)\n",
    "alice, bob = sf.PYU('alice'), sf.PYU('bob')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### prepare data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from secretflow.data.simulation.dataset import load_bank_marketing_data\n",
    "\n",
    "column_split = {\n",
    "            alice: [\"age\",\"job\",\"marital\",\"education\",\"y\"],\n",
    "            bob: [\"default\",\"balance\",\"housing\",\"loan\",\"contact\",\"day\",\"month\",\"duration\",\"campaign\",\"pdays\",\"previous\",\"poutcome\"],\n",
    "        }\n",
    "file_uris = load_bank_marketing_data(party_ratio=column_split)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "dataset_dict = {}\n",
    "for device, file_path in file_uris.items():\n",
    "    dataset_dict[device] = pd.read_csv(file_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We assume that Alice is a new bank, and they only have the basic information of the user and purchased the label of financial products from other bank."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>age</th>\n",
       "      <th>job</th>\n",
       "      <th>marital</th>\n",
       "      <th>education</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>30</td>\n",
       "      <td>unemployed</td>\n",
       "      <td>married</td>\n",
       "      <td>primary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>33</td>\n",
       "      <td>services</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>35</td>\n",
       "      <td>management</td>\n",
       "      <td>single</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>30</td>\n",
       "      <td>management</td>\n",
       "      <td>married</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>59</td>\n",
       "      <td>blue-collar</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4516</th>\n",
       "      <td>4516</td>\n",
       "      <td>33</td>\n",
       "      <td>services</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4517</th>\n",
       "      <td>4517</td>\n",
       "      <td>57</td>\n",
       "      <td>self-employed</td>\n",
       "      <td>married</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4518</th>\n",
       "      <td>4518</td>\n",
       "      <td>57</td>\n",
       "      <td>technician</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4519</th>\n",
       "      <td>4519</td>\n",
       "      <td>28</td>\n",
       "      <td>blue-collar</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4520</th>\n",
       "      <td>4520</td>\n",
       "      <td>44</td>\n",
       "      <td>entrepreneur</td>\n",
       "      <td>single</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4521 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        id  age            job  marital  education   y\n",
       "0        0   30     unemployed  married    primary  no\n",
       "1        1   33       services  married  secondary  no\n",
       "2        2   35     management   single   tertiary  no\n",
       "3        3   30     management  married   tertiary  no\n",
       "4        4   59    blue-collar  married  secondary  no\n",
       "...    ...  ...            ...      ...        ...  ..\n",
       "4516  4516   33       services  married  secondary  no\n",
       "4517  4517   57  self-employed  married   tertiary  no\n",
       "4518  4518   57     technician  married  secondary  no\n",
       "4519  4519   28    blue-collar  married  secondary  no\n",
       "4520  4520   44   entrepreneur   single   tertiary  no\n",
       "\n",
       "[4521 rows x 6 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset_dict[alice]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.frame.DataFrame"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(dataset_dict[alice])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bob is an old bank, they have the user's account balance, house, loan, and recent marketing feedback"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>default</th>\n",
       "      <th>balance</th>\n",
       "      <th>housing</th>\n",
       "      <th>loan</th>\n",
       "      <th>contact</th>\n",
       "      <th>day</th>\n",
       "      <th>month</th>\n",
       "      <th>duration</th>\n",
       "      <th>campaign</th>\n",
       "      <th>pdays</th>\n",
       "      <th>previous</th>\n",
       "      <th>poutcome</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>no</td>\n",
       "      <td>1787</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>cellular</td>\n",
       "      <td>19</td>\n",
       "      <td>oct</td>\n",
       "      <td>79</td>\n",
       "      <td>1</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>no</td>\n",
       "      <td>4789</td>\n",
       "      <td>yes</td>\n",
       "      <td>yes</td>\n",
       "      <td>cellular</td>\n",
       "      <td>11</td>\n",
       "      <td>may</td>\n",
       "      <td>220</td>\n",
       "      <td>1</td>\n",
       "      <td>339</td>\n",
       "      <td>4</td>\n",
       "      <td>failure</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>no</td>\n",
       "      <td>1350</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>cellular</td>\n",
       "      <td>16</td>\n",
       "      <td>apr</td>\n",
       "      <td>185</td>\n",
       "      <td>1</td>\n",
       "      <td>330</td>\n",
       "      <td>1</td>\n",
       "      <td>failure</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>no</td>\n",
       "      <td>1476</td>\n",
       "      <td>yes</td>\n",
       "      <td>yes</td>\n",
       "      <td>unknown</td>\n",
       "      <td>3</td>\n",
       "      <td>jun</td>\n",
       "      <td>199</td>\n",
       "      <td>4</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>unknown</td>\n",
       "      <td>5</td>\n",
       "      <td>may</td>\n",
       "      <td>226</td>\n",
       "      <td>1</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4516</th>\n",
       "      <td>4516</td>\n",
       "      <td>no</td>\n",
       "      <td>-333</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>cellular</td>\n",
       "      <td>30</td>\n",
       "      <td>jul</td>\n",
       "      <td>329</td>\n",
       "      <td>5</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4517</th>\n",
       "      <td>4517</td>\n",
       "      <td>yes</td>\n",
       "      <td>-3313</td>\n",
       "      <td>yes</td>\n",
       "      <td>yes</td>\n",
       "      <td>unknown</td>\n",
       "      <td>9</td>\n",
       "      <td>may</td>\n",
       "      <td>153</td>\n",
       "      <td>1</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4518</th>\n",
       "      <td>4518</td>\n",
       "      <td>no</td>\n",
       "      <td>295</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>cellular</td>\n",
       "      <td>19</td>\n",
       "      <td>aug</td>\n",
       "      <td>151</td>\n",
       "      <td>11</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4519</th>\n",
       "      <td>4519</td>\n",
       "      <td>no</td>\n",
       "      <td>1137</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>cellular</td>\n",
       "      <td>6</td>\n",
       "      <td>feb</td>\n",
       "      <td>129</td>\n",
       "      <td>4</td>\n",
       "      <td>211</td>\n",
       "      <td>3</td>\n",
       "      <td>other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4520</th>\n",
       "      <td>4520</td>\n",
       "      <td>no</td>\n",
       "      <td>1136</td>\n",
       "      <td>yes</td>\n",
       "      <td>yes</td>\n",
       "      <td>cellular</td>\n",
       "      <td>3</td>\n",
       "      <td>apr</td>\n",
       "      <td>345</td>\n",
       "      <td>2</td>\n",
       "      <td>249</td>\n",
       "      <td>7</td>\n",
       "      <td>other</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4521 rows × 13 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        id default  balance housing loan   contact  day month  duration  \\\n",
       "0        0      no     1787      no   no  cellular   19   oct        79   \n",
       "1        1      no     4789     yes  yes  cellular   11   may       220   \n",
       "2        2      no     1350     yes   no  cellular   16   apr       185   \n",
       "3        3      no     1476     yes  yes   unknown    3   jun       199   \n",
       "4        4      no        0     yes   no   unknown    5   may       226   \n",
       "...    ...     ...      ...     ...  ...       ...  ...   ...       ...   \n",
       "4516  4516      no     -333     yes   no  cellular   30   jul       329   \n",
       "4517  4517     yes    -3313     yes  yes   unknown    9   may       153   \n",
       "4518  4518      no      295      no   no  cellular   19   aug       151   \n",
       "4519  4519      no     1137      no   no  cellular    6   feb       129   \n",
       "4520  4520      no     1136     yes  yes  cellular    3   apr       345   \n",
       "\n",
       "      campaign  pdays  previous poutcome  \n",
       "0            1     -1         0  unknown  \n",
       "1            1    339         4  failure  \n",
       "2            1    330         1  failure  \n",
       "3            4     -1         0  unknown  \n",
       "4            1     -1         0  unknown  \n",
       "...        ...    ...       ...      ...  \n",
       "4516         5     -1         0  unknown  \n",
       "4517         1     -1         0  unknown  \n",
       "4518        11     -1         0  unknown  \n",
       "4519         4    211         3    other  \n",
       "4520         2    249         7    other  \n",
       "\n",
       "[4521 rows x 13 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset_dict[bob]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create Secretflow Environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create 2 entities in the Secretflow environment [Alice, Bob]\n",
    "Where 'Alice' and 'Bob' are two PYU\n",
    "Once you've constructed the two objects, you can happily start Splitting Learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import Dependency"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from secretflow.data.split import train_test_split\n",
    "from secretflow.ml.nn import SLModelTF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Build Federated Table**  \n",
    "Federated table is a virtual concept that cross multiple parties,We define `VDataFrame` for vertical setting\n",
    "1. The data of all parties in a federated table is stored locally and is not allowed to go out of the domain\n",
    "2. No one has access to data store except the party that owns the data\n",
    "3. Any operation of the federated table will be scheduled by the driver to each worker, and the execution instructions will be delivered layer by layer until the Python Runtime of the specific worker. The framework ensures that only `worker.device` and `Object`. device can operate data at the same time.。\n",
    "4. Federated tables are designed to management and manipulation multi-party data from a central perspective\n",
    "5. Interfaces to `Federated Tables` are aligned to pandas.DataFrame to reduce the cost of multi-party data operations\n",
    "6. The SecretFlow framework provides Plain&Ciphertext hybrid programming capabilities. Vertical federated tables are built using `SPU`, and `Mpc-psi` is used to safely get intersection and align data from all parties\n",
    "\n",
    "<img alt=\"vdataframe.png\" src=\"resource/vdataframe.png\" width=\"600\">  \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "VDataFrame provides `read_csv` interface similar to pandas, except that `secretflow.read_csv` receives a dictionary that defines the path of data for both parties. We can use `secretflow.vertical.read_csv` to build the `VDataFrame`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create spu object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(pid=2230314)\u001b[0m 2022-06-29 11:39:41.097060: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib:/opt/rh/devtoolset-10/root/usr/lib64/dyninst:/opt/rh/devtoolset-10/root/usr/lib/dyninst:/opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib\n",
      "\u001b[2m\u001b[36m(pid=2230315)\u001b[0m 2022-06-29 11:39:41.097060: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib:/opt/rh/devtoolset-10/root/usr/lib64/dyninst:/opt/rh/devtoolset-10/root/usr/lib/dyninst:/opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib\n"
     ]
    }
   ],
   "source": [
    "spu = sf.SPU(sf.utils.testing.cluster_def(['alice', 'bob']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(SPURuntime pid=2230314)\u001b[0m I0629 11:39:42.788682 2230314 external/com_github_brpc_brpc/src/brpc/server.cpp:1065] Server[yasl::link::internal::ReceiverServiceImpl] is serving on port=15261.\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=2230314)\u001b[0m I0629 11:39:42.788750 2230314 external/com_github_brpc_brpc/src/brpc/server.cpp:1068] Check out http://i85c08157.eu95sqa:15261 in web browser.\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=2230315)\u001b[0m I0629 11:39:42.790211 2230315 external/com_github_brpc_brpc/src/brpc/server.cpp:1065] Server[yasl::link::internal::ReceiverServiceImpl] is serving on port=50563.\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=2230315)\u001b[0m I0629 11:39:42.790271 2230315 external/com_github_brpc_brpc/src/brpc/server.cpp:1068] Check out http://i85c08157.eu95sqa:50563 in web browser.\n",
      "\u001b[2m\u001b[36m(_run pid=2230316)\u001b[0m 2022-06-29 11:39:43.080818: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib:/opt/rh/devtoolset-10/root/usr/lib64/dyninst:/opt/rh/devtoolset-10/root/usr/lib/dyninst:/opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib\n",
      "\u001b[2m\u001b[36m(_run pid=2230318)\u001b[0m 2022-06-29 11:39:43.080972: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib:/opt/rh/devtoolset-10/root/usr/lib64/dyninst:/opt/rh/devtoolset-10/root/usr/lib/dyninst:/opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(SPURuntime pid=2230315)\u001b[0m [2022-06-29 11:39:44.808] [info] [executor_base.cc:246] Begin sanity check for input file: .data/1/psi-input.csv\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=2230315)\u001b[0m [2022-06-29 11:39:44.811] [info] [executor_base.cc:196] Executing duplicated scripts: LC_ALL=C sort --buffer-size=1G --temporary-directory=./ --stable .data/1/psi-input.csv.keys.1656473984808312593 | LC_ALL=C uniq -d > .data/1/psi-input.csv.duplicated.1656473984808312593\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=2230315)\u001b[0m [2022-06-29 11:39:44.814] [info] [executor_base.cc:199] Finished duplicated scripts: LC_ALL=C sort --buffer-size=1G --temporary-directory=./ --stable .data/1/psi-input.csv.keys.1656473984808312593 | LC_ALL=C uniq -d > .data/1/psi-input.csv.duplicated.1656473984808312593, ret=0\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=2230315)\u001b[0m [2022-06-29 11:39:44.814] [info] [executor_base.cc:249] End sanity check for input file: .data/1/psi-input.csv, size=4521\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=2230315)\u001b[0m [2022-06-29 11:39:44.821] [info] [executor_base.cc:262] skip doing psi, because dataset has been aligned!\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=2230315)\u001b[0m [2022-06-29 11:39:44.821] [info] [executor_base.cc:268] Begin post filtering, indices.size=4521, should_sort=true\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=2230315)\u001b[0m [2022-06-29 11:39:44.823] [info] [executor_base.cc:287] End post filtering, in=.data/1/psi-input.csv, out=.data/1/psi-output.csv\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=2230314)\u001b[0m [2022-06-29 11:39:44.815] [info] [executor_base.cc:246] Begin sanity check for input file: .data/0/psi-input.csv\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=2230314)\u001b[0m [2022-06-29 11:39:44.818] [info] [executor_base.cc:196] Executing duplicated scripts: LC_ALL=C sort --buffer-size=1G --temporary-directory=./ --stable .data/0/psi-input.csv.keys.1656473984815739130 | LC_ALL=C uniq -d > .data/0/psi-input.csv.duplicated.1656473984815739130\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=2230314)\u001b[0m [2022-06-29 11:39:44.821] [info] [executor_base.cc:199] Finished duplicated scripts: LC_ALL=C sort --buffer-size=1G --temporary-directory=./ --stable .data/0/psi-input.csv.keys.1656473984815739130 | LC_ALL=C uniq -d > .data/0/psi-input.csv.duplicated.1656473984815739130, ret=0\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=2230314)\u001b[0m [2022-06-29 11:39:44.821] [info] [executor_base.cc:249] End sanity check for input file: .data/0/psi-input.csv, size=4521\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=2230314)\u001b[0m [2022-06-29 11:39:44.821] [info] [executor_base.cc:262] skip doing psi, because dataset has been aligned!\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=2230314)\u001b[0m [2022-06-29 11:39:44.821] [info] [executor_base.cc:268] Begin post filtering, indices.size=4521, should_sort=true\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=2230314)\u001b[0m [2022-06-29 11:39:44.823] [info] [executor_base.cc:287] End post filtering, in=.data/0/psi-input.csv, out=.data/0/psi-output.csv\n"
     ]
    }
   ],
   "source": [
    "from secretflow.data.vertical import read_csv\n",
    "\n",
    "vdf = read_csv(file_uris,spu=spu,keys='id',drop_keys=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`VDF` is a vertically federated table that has been built. It has only the `Schema` of all the data globally"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['age', 'job', 'marital', 'education', 'y', 'default', 'balance',\n",
       "       'housing', 'loan', 'contact', 'day', 'month', 'duration', 'campaign',\n",
       "       'pdays', 'previous', 'poutcome'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vdf.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a closer look at VDF data management \n",
    "\n",
    "As can be seen from an example, the `age` field belongs to Alice, so the corresponding column can be obtained in the partition of Alice, but Bob will report `KeyError` error when trying to obtain age.  \n",
    "There is a concept of `Partition`, which is a data fragment defined by us. Each Partition has its own device to which it belongs, and only the device that belongs can operate data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<secretflow.device.device.pyu.PYUObject object at 0x7f06a6751a30>\n"
     ]
    },
    {
     "ename": "KeyError",
     "evalue": "<secretflow.device.device.pyu.PYU object at 0x7f06e82c04f0>",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "Input \u001b[0;32mIn [11]\u001b[0m, in \u001b[0;36m<cell line: 2>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[38;5;28mprint\u001b[39m(vdf[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mage\u001b[39m\u001b[38;5;124m'\u001b[39m]\u001b[38;5;241m.\u001b[39mpartitions[alice]\u001b[38;5;241m.\u001b[39mdata)\n\u001b[0;32m----> 2\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[43mvdf\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mage\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m]\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mpartitions\u001b[49m\u001b[43m[\u001b[49m\u001b[43mbob\u001b[49m\u001b[43m]\u001b[49m)\n",
      "\u001b[0;31mKeyError\u001b[0m: <secretflow.device.device.pyu.PYU object at 0x7f06e82c04f0>"
     ]
    }
   ],
   "source": [
    "print(vdf['age'].partitions[alice].data)\n",
    "print(vdf['age'].partitions[bob])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We then do data preprocessing on the `VDataFrame`.。  \n",
    "Here we take `LabelEncoder` and `MinMaxScaler` as examples. These two preprocessor functions have corresponding concepts in `SkLearn` and their use methods are similar to those in skLearn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from secretflow.preprocessing.scaler import MinMaxScaler\n",
    "from secretflow.preprocessing.encoder import LabelEncoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "encoder = LabelEncoder()\n",
    "vdf['job'] = encoder.fit_transform(vdf['job'])\n",
    "vdf['marital'] = encoder.fit_transform(vdf['marital'])\n",
    "vdf['education'] = encoder.fit_transform(vdf['education'])\n",
    "vdf['default'] = encoder.fit_transform(vdf['default'])\n",
    "vdf['housing'] = encoder.fit_transform(vdf['housing'])\n",
    "vdf['loan'] = encoder.fit_transform(vdf['loan'])\n",
    "vdf['contact'] = encoder.fit_transform(vdf['contact'])\n",
    "vdf['poutcome'] = encoder.fit_transform(vdf['poutcome'])\n",
    "vdf['month'] = encoder.fit_transform(vdf['month'])\n",
    "vdf['y'] = encoder.fit_transform(vdf['y'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We split the data into data and label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "label = vdf['y']\n",
    "data = vdf.drop(columns='y', inplace=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "label= <class 'secretflow.data.vertical.dataframe.VDataFrame'>,\n",
      "data = <class 'secretflow.data.vertical.dataframe.VDataFrame'>\n"
     ]
    }
   ],
   "source": [
    "print(f\"label= {type(label)},\\ndata = {type(data)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Data standardization via MinMaxScaler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(_run pid=2230316)\u001b[0m /home/xingmeng.zhxm/anaconda3/envs/secretflow/lib/python3.8/site-packages/sklearn/base.py:443: UserWarning: X has feature names, but MinMaxScaler was fitted without feature names\n",
      "\u001b[2m\u001b[36m(_run pid=2230316)\u001b[0m   warnings.warn(\n",
      "\u001b[2m\u001b[36m(_run pid=2230318)\u001b[0m /home/xingmeng.zhxm/anaconda3/envs/secretflow/lib/python3.8/site-packages/sklearn/base.py:443: UserWarning: X has feature names, but MinMaxScaler was fitted without feature names\n",
      "\u001b[2m\u001b[36m(_run pid=2230318)\u001b[0m   warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "scaler = MinMaxScaler()\n",
    "\n",
    "data = scaler.fit_transform(vdf[list(data.columns)])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we divide the data set into train-set and test-set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from secretflow.data.split import train_test_split\n",
    "random_state = 1234\n",
    "train_data,test_data = train_test_split(data,train_size=0.8,random_state=random_state)\n",
    "train_label,test_label = train_test_split(label,train_size=0.8,random_state=random_state)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Summary:** At this point, we have completed the definition of **federated tables**, **data preprocessing**, and **training set and test set partitioning**\n",
    "The secretFlow framework defines a set of operations to be built on the federated table (its logical counterpart is `pandas.DataFrame`). The secretflow framework defines a set of operations to be built on the federated table (its logical counterpart is `sklearn`) Refer to our documentation and API introduction to learn more about other features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduce Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**local version**: \n",
    "For this task, a basic DNN can be completed, input 16-dimensional features, through a DNN network, output the probability of positive and negative samples.\n",
    "\n",
    "\n",
    "**Federate version**：\n",
    "* Alice：\n",
    "    - base_net: Input 4-dimensional feature and go through a DNN network to get hidden\n",
    "    - fuse_net: Receive hidden features calculated by Alice and Bob, input them to FUSENET for feature fusion, and complete the forward process and backward process\n",
    "* Bob：\n",
    "    - base_net: Input 12-dimensional features, get hidden through a DNN network, and then send hidden to Alice to complete the following operation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we start creating the federated model \n",
    "we define SLTFModel and SLTorchModel(WIP), which are used to build split learning of vertical scene. We define a simple and easy to use extensible interface, which can easily transform your existing Model into SF-Model, and then conduct vertical scene federation modeling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Split learning is to break up a model so that one part is executed locally on the data and the other part is executed on the label side.\n",
    "First let's define the locally executed model -- base_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_base_model(input_dim, output_dim,  name='base_model'):\n",
    "    # Create model\n",
    "    def create_model():\n",
    "        from tensorflow import keras\n",
    "        from tensorflow.keras import layers\n",
    "        import tensorflow as tf\n",
    "        model = keras.Sequential(\n",
    "            [\n",
    "                keras.Input(shape=input_dim),\n",
    "                layers.Dense(100,activation =\"relu\" ),\n",
    "                layers.Dense(output_dim, activation=\"relu\"),\n",
    "            ]\n",
    "        )\n",
    "        # Compile model\n",
    "        model.summary()\n",
    "        model.compile(loss='binary_crossentropy',\n",
    "                      optimizer='adam',\n",
    "                      metrics=[\"accuracy\",tf.keras.metrics.AUC()])\n",
    "        return model\n",
    "    return create_model\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use create_base_model to create their base models for 'Alice' and 'Bob', respectively"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# prepare model\n",
    "hidden_size = 64\n",
    "\n",
    "model_base_alice = create_base_model(4, hidden_size)\n",
    "model_base_bob = create_base_model(12, hidden_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " dense (Dense)               (None, 100)               500       \n",
      "                                                                 \n",
      " dense_1 (Dense)             (None, 64)                6464      \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 6,964\n",
      "Trainable params: 6,964\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " dense_2 (Dense)             (None, 100)               1300      \n",
      "                                                                 \n",
      " dense_3 (Dense)             (None, 64)                6464      \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 7,764\n",
      "Trainable params: 7,764\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-06-29 11:40:23.180851: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib:/opt/rh/devtoolset-10/root/usr/lib64/dyninst:/opt/rh/devtoolset-10/root/usr/lib/dyninst:/opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib\n",
      "2022-06-29 11:40:23.180895: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.engine.sequential.Sequential at 0x7f06a5c1bdf0>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_base_alice()\n",
    "model_base_bob()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we define the side with the label, or the server-side model -- fuse_model\n",
    "In the definition of fuse_model, we need to correctly define `loss`, `optimizer`, and `metrics`. This is compatible with all configurations of your existing Keras model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_fuse_model(input_dim, output_dim, party_nums, name='fuse_model'):\n",
    "    def create_model():\n",
    "        from tensorflow import keras\n",
    "        from tensorflow.keras import layers\n",
    "        import tensorflow as tf\n",
    "        # input\n",
    "        input_layers = []\n",
    "        for i in range(party_nums):\n",
    "            input_layers.append(keras.Input(input_dim,))\n",
    "        \n",
    "        merged_layer = layers.concatenate(input_layers)\n",
    "        fuse_layer = layers.Dense(64, activation='relu')(merged_layer)\n",
    "        output = layers.Dense(output_dim, activation='sigmoid')(fuse_layer)\n",
    "\n",
    "        model = keras.Model(inputs=input_layers, outputs=output)\n",
    "        model.summary()\n",
    "        \n",
    "        model.compile(loss='binary_crossentropy',\n",
    "                      optimizer='adam',\n",
    "                      metrics=[\"accuracy\",tf.keras.metrics.AUC()])\n",
    "        return model\n",
    "    return create_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_fuse = create_fuse_model(\n",
    "    input_dim=hidden_size, party_nums=2, output_dim=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model\"\n",
      "__________________________________________________________________________________________________\n",
      " Layer (type)                   Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      " input_3 (InputLayer)           [(None, 64)]         0           []                               \n",
      "                                                                                                  \n",
      " input_4 (InputLayer)           [(None, 64)]         0           []                               \n",
      "                                                                                                  \n",
      " concatenate (Concatenate)      (None, 128)          0           ['input_3[0][0]',                \n",
      "                                                                  'input_4[0][0]']                \n",
      "                                                                                                  \n",
      " dense_4 (Dense)                (None, 64)           8256        ['concatenate[0][0]']            \n",
      "                                                                                                  \n",
      " dense_5 (Dense)                (None, 1)            65          ['dense_4[0][0]']                \n",
      "                                                                                                  \n",
      "==================================================================================================\n",
      "Total params: 8,321\n",
      "Trainable params: 8,321\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.engine.functional.Functional at 0x7f06a4aa83a0>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_fuse()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create Split Learning Model\n",
    "Secretflow provides the split learning model——SLModelTF\n",
    "To initial SLModelTF only need 3 parameters\n",
    "* base_model_dict：A dictionary needs to be passed in all clients participating in the training along with base_model mappings\n",
    "* device_y：PYU，which device has label\n",
    "* model_fuse：The fusion model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define base_model_dict  \n",
    "```python\n",
    "base_model_dict:Dict[PYU,model_fn]\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "base_model_dict = {\n",
    "    alice: model_base_alice,\n",
    "    bob:   model_base_bob\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "from secretflow.security.privacy import DPStrategy, GaussianEmbeddingDP, LabelDP\n",
    "\n",
    "# Define DP operations\n",
    "train_batch_size = 128\n",
    "gaussian_embedding_dp = GaussianEmbeddingDP(\n",
    "    noise_multiplier=0.5,\n",
    "    l2_norm_clip=1.0,\n",
    "    batch_size=train_batch_size,\n",
    "    num_samples=train_data.values.partition_shape()[alice][0],\n",
    "    is_secure_generator=False,\n",
    ")\n",
    "dp_strategy_alice = DPStrategy(embedding_dp=gaussian_embedding_dp)\n",
    "label_dp = LabelDP(eps=64.0)\n",
    "dp_strategy_bob = DPStrategy(label_dp=label_dp)\n",
    "dp_strategy_dict = {alice: dp_strategy_alice, bob: dp_strategy_bob}\n",
    "dp_spent_step_freq = 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "sl_model = SLModelTF(\n",
    "    base_model_dict=base_model_dict, \n",
    "    device_y=alice,  \n",
    "    model_fuse=model_fuse,\n",
    "    dp_strategy_dict=dp_strategy_dict,)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m 2022-06-29 11:40:28.379543: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib:/opt/rh/devtoolset-10/root/usr/lib64/dyninst:/opt/rh/devtoolset-10/root/usr/lib/dyninst:/opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m 2022-06-29 11:40:28.379573: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m 2022-06-29 11:40:28.381398: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib:/opt/rh/devtoolset-10/root/usr/lib64/dyninst:/opt/rh/devtoolset-10/root/usr/lib/dyninst:/opt/rh/devtoolset-10/root/usr/lib64:/opt/rh/devtoolset-10/root/usr/lib\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m 2022-06-29 11:40:28.381424: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)\n",
      "2022-06-29 11:40:28.484 | DEBUG    | secretflow.ml.nn.sl_model:fit:167 - validation_data provided\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m Model: \"sequential\"\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m _________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m  Layer (type)                Output Shape              Param #   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m =================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m  dense (Dense)               (None, 100)               500       \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m                                                                  \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m  dense_1 (Dense)             (None, 64)                6464      \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m                                                                  \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m =================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m Total params: 6,964\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m Trainable params: 6,964\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m Non-trainable params: 0\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m _________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m Model: \"sequential\"\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m _________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m  Layer (type)                Output Shape              Param #   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m =================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m  dense (Dense)               (None, 100)               1300      \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m                                                                  \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m  dense_1 (Dense)             (None, 64)                6464      \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m                                                                  \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m =================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m Total params: 7,764\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m Trainable params: 7,764\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m Non-trainable params: 0\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m _________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m Model: \"sequential_1\"\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m _________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m  Layer (type)                Output Shape              Param #   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m =================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m  dense_2 (Dense)             (None, 100)               500       \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m                                                                  \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m  dense_3 (Dense)             (None, 64)                6464      \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m                                                                  \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m =================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m Total params: 6,964\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m Trainable params: 6,964\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m Non-trainable params: 0\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m _________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m Model: \"model\"\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m __________________________________________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m  Layer (type)                   Output Shape         Param #     Connected to                     \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m ==================================================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m  input_3 (InputLayer)           [(None, 64)]         0           []                               \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m                                                                                                   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m  input_4 (InputLayer)           [(None, 64)]         0           []                               \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m                                                                                                   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m  concatenate (Concatenate)      (None, 128)          0           ['input_3[0][0]',                \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m                                                                   'input_4[0][0]']                \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m                                                                                                   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m  dense_4 (Dense)                (None, 64)           8256        ['concatenate[0][0]']            \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m                                                                                                   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m  dense_5 (Dense)                (None, 1)            65          ['dense_4[0][0]']                \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m                                                                                                   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m ==================================================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m Total params: 8,321\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m Trainable params: 8,321\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m Non-trainable params: 0\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230316)\u001b[0m __________________________________________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m Model: \"sequential_1\"\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m _________________________________________________________________\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m  Layer (type)                Output Shape              Param #   \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m =================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m  dense_2 (Dense)             (None, 100)               1300      \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m                                                                  \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m  dense_3 (Dense)             (None, 64)                6464      \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m                                                                  \n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m =================================================================\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m Total params: 7,764\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m Trainable params: 7,764\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m Non-trainable params: 0\n",
      "\u001b[2m\u001b[36m(PYUSLTFModel pid=2230318)\u001b[0m _________________________________________________________________\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 29/29 [00:01<00:00, 18.39it/s, epoch: 0/10 -  train_loss:0.42552024126052856  train_accuracy:0.8590909242630005  train_auc_2:0.5520472526550293  val_loss:0.34310784935951233  val_accuracy:0.8877212405204773  val_auc_2:0.6612115502357483 ]\n",
      "100%|██████████| 29/29 [00:01<00:00, 19.32it/s, epoch: 1/10 -  train_loss:0.33578866720199585  train_accuracy:0.8902838230133057  train_auc_2:0.6446549892425537  val_loss:0.3264640271663666  val_accuracy:0.8877212405204773  val_auc_2:0.7030531167984009 ]\n",
      "100%|██████████| 29/29 [00:01<00:00, 19.05it/s, epoch: 2/10 -  train_loss:0.3369784355163574  train_accuracy:0.8823689818382263  train_auc_2:0.700323760509491  val_loss:0.31710633635520935  val_accuracy:0.8877212405204773  val_auc_2:0.7425126433372498 ]\n",
      "100%|██████████| 29/29 [00:01<00:00, 19.38it/s, epoch: 3/10 -  train_loss:0.31362512707710266  train_accuracy:0.8870298862457275  train_auc_2:0.751912534236908  val_loss:0.2964775562286377  val_accuracy:0.8877212405204773  val_auc_2:0.7922881841659546 ]\n",
      "100%|██████████| 29/29 [00:01<00:00, 20.78it/s, epoch: 4/10 -  train_loss:0.2956601083278656  train_accuracy:0.8855100870132446  train_auc_2:0.8006429672241211  val_loss:0.2768186628818512  val_accuracy:0.888550877571106  val_auc_2:0.8328522443771362 ]\n",
      "100%|██████████| 29/29 [00:01<00:00, 21.34it/s, epoch: 5/10 -  train_loss:0.2666192948818207  train_accuracy:0.8909753561019897  train_auc_2:0.8438374400138855  val_loss:0.257427453994751  val_accuracy:0.894911527633667  val_auc_2:0.8583360314369202 ]\n",
      "100%|██████████| 29/29 [00:01<00:00, 21.15it/s, epoch: 6/10 -  train_loss:0.25582611560821533  train_accuracy:0.8939683437347412  train_auc_2:0.8628933429718018  val_loss:0.258818119764328  val_accuracy:0.8946349620819092  val_auc_2:0.8741072416305542 ]\n",
      "100%|██████████| 29/29 [00:01<00:00, 20.90it/s, epoch: 7/10 -  train_loss:0.250174343585968  train_accuracy:0.8987832069396973  train_auc_2:0.8658520579338074  val_loss:0.24163179099559784  val_accuracy:0.8985066413879395  val_auc_2:0.8827099204063416 ]\n",
      "100%|██████████| 29/29 [00:01<00:00, 21.06it/s, epoch: 8/10 -  train_loss:0.24141278862953186  train_accuracy:0.8996128439903259  train_auc_2:0.875605583190918  val_loss:0.24190263450145721  val_accuracy:0.8993362784385681  val_auc_2:0.8805226683616638 ]\n",
      "100%|██████████| 29/29 [00:01<00:00, 20.86it/s, epoch: 9/10 -  train_loss:0.23765592277050018  train_accuracy:0.9034845232963562  train_auc_2:0.8746204376220703  val_loss:0.24357406795024872  val_accuracy:0.9004424810409546  val_auc_2:0.8792129158973694 ]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'train_loss': [0.42552024,\n",
       "  0.33578867,\n",
       "  0.33697844,\n",
       "  0.31362513,\n",
       "  0.2956601,\n",
       "  0.2666193,\n",
       "  0.25582612,\n",
       "  0.25017434,\n",
       "  0.24141279,\n",
       "  0.23765592],\n",
       " 'train_accuracy': [0.8590909,\n",
       "  0.8902838,\n",
       "  0.882369,\n",
       "  0.8870299,\n",
       "  0.8855101,\n",
       "  0.89097536,\n",
       "  0.89396834,\n",
       "  0.8987832,\n",
       "  0.89961284,\n",
       "  0.9034845],\n",
       " 'train_auc_2': [0.55204725,\n",
       "  0.644655,\n",
       "  0.70032376,\n",
       "  0.75191253,\n",
       "  0.80064297,\n",
       "  0.84383744,\n",
       "  0.86289334,\n",
       "  0.86585206,\n",
       "  0.8756056,\n",
       "  0.87462044],\n",
       " 'val_loss': [0.34310785,\n",
       "  0.32646403,\n",
       "  0.31710634,\n",
       "  0.29647756,\n",
       "  0.27681866,\n",
       "  0.25742745,\n",
       "  0.25881812,\n",
       "  0.24163179,\n",
       "  0.24190263,\n",
       "  0.24357407],\n",
       " 'val_accuracy': [0.88772124,\n",
       "  0.88772124,\n",
       "  0.88772124,\n",
       "  0.88772124,\n",
       "  0.8885509,\n",
       "  0.8949115,\n",
       "  0.89463496,\n",
       "  0.89850664,\n",
       "  0.8993363,\n",
       "  0.9004425],\n",
       " 'val_auc_2': [0.66121155,\n",
       "  0.7030531,\n",
       "  0.74251264,\n",
       "  0.7922882,\n",
       "  0.83285224,\n",
       "  0.85833603,\n",
       "  0.87410724,\n",
       "  0.8827099,\n",
       "  0.88052267,\n",
       "  0.8792129]}"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sl_model.fit(train_data,\n",
    "             train_label,\n",
    "             validation_data=(test_data,test_label),\n",
    "             epochs=10, \n",
    "             batch_size=train_batch_size,\n",
    "             shuffle=True,\n",
    "             verbose=1,\n",
    "             validation_freq=1,\n",
    "            dp_spent_step_freq=dp_spent_step_freq,)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's call the evaluation function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Evaluate Processing:: 100%|██████████| 29/29 [00:00<00:00, 62.88it/s, loss:0.24504481256008148 accuracy:0.8976770043373108 auc_2:0.8779380321502686]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 0.24504481, 'accuracy': 0.897677, 'auc_2': 0.87793803}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "global_metric = sl_model.evaluate(test_data, test_label, batch_size=128)\n",
    "print(global_metric)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contrast to local model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Model\n",
    "The model structure is consistent with the model of split learning above, but only the model structure of Alice is used here. The model definition refers to the code below.\n",
    "#### Data\n",
    "The data also use kaggle's anti-fraud data. Here, we just use Alice's data of the new bank.\n",
    "1. The total sample size was 11162, including 8929 training set and 2233 test set.\n",
    "2. The feature dimension is 4."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "import tensorflow as tf\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "def create_model():\n",
    "\n",
    "    model = keras.Sequential(\n",
    "        [\n",
    "            keras.Input(shape=4),\n",
    "            layers.Dense(100,activation =\"relu\" ),\n",
    "            layers.Dense(64, activation='relu'),\n",
    "            layers.Dense(64, activation='relu'),\n",
    "            layers.Dense(1, activation='sigmoid')\n",
    "        ]\n",
    "    )\n",
    "    model.compile(loss='binary_crossentropy',\n",
    "                      optimizer='adam',\n",
    "                      metrics=[\"accuracy\",tf.keras.metrics.AUC()])\n",
    "    return model\n",
    "\n",
    "single_model = create_model()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "data process"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset_dict[alice]= dataset_dict[alice].drop(columns=\"id\",inplace=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>job</th>\n",
       "      <th>marital</th>\n",
       "      <th>education</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>30</td>\n",
       "      <td>unemployed</td>\n",
       "      <td>married</td>\n",
       "      <td>primary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>33</td>\n",
       "      <td>services</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>35</td>\n",
       "      <td>management</td>\n",
       "      <td>single</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>30</td>\n",
       "      <td>management</td>\n",
       "      <td>married</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>59</td>\n",
       "      <td>blue-collar</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4516</th>\n",
       "      <td>33</td>\n",
       "      <td>services</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4517</th>\n",
       "      <td>57</td>\n",
       "      <td>self-employed</td>\n",
       "      <td>married</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4518</th>\n",
       "      <td>57</td>\n",
       "      <td>technician</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4519</th>\n",
       "      <td>28</td>\n",
       "      <td>blue-collar</td>\n",
       "      <td>married</td>\n",
       "      <td>secondary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4520</th>\n",
       "      <td>44</td>\n",
       "      <td>entrepreneur</td>\n",
       "      <td>single</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4521 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      age            job  marital  education   y\n",
       "0      30     unemployed  married    primary  no\n",
       "1      33       services  married  secondary  no\n",
       "2      35     management   single   tertiary  no\n",
       "3      30     management  married   tertiary  no\n",
       "4      59    blue-collar  married  secondary  no\n",
       "...   ...            ...      ...        ...  ..\n",
       "4516   33       services  married  secondary  no\n",
       "4517   57  self-employed  married   tertiary  no\n",
       "4518   57     technician  married  secondary  no\n",
       "4519   28    blue-collar  married  secondary  no\n",
       "4520   44   entrepreneur   single   tertiary  no\n",
       "\n",
       "[4521 rows x 5 columns]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset_dict[alice]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "alice_data = dataset_dict[alice]\n",
    "encoder = LabelEncoder()\n",
    "alice_data['job'] = encoder.fit_transform(alice_data['job'])\n",
    "alice_data['marital'] = encoder.fit_transform(alice_data['marital'])\n",
    "alice_data['education'] = encoder.fit_transform(alice_data['education'])\n",
    "alice_data['y'] =  encoder.fit_transform(alice_data['y'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = alice_data['y']\n",
    "alice_data = alice_data.drop(columns=['y'],inplace=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = MinMaxScaler()\n",
    "alice_data = scaler.fit_transform(alice_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data,test_data = train_test_split(alice_data,train_size=0.8,random_state=random_state)\n",
    "train_label,test_label = train_test_split(y,train_size=0.8,random_state=random_state)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(905, 4)"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "29/29 [==============================] - 1s 13ms/step - loss: 0.5368 - accuracy: 0.8097 - auc_3: 0.4534 - val_loss: 0.4007 - val_accuracy: 0.8729 - val_auc_3: 0.4284\n",
      "Epoch 2/10\n",
      "29/29 [==============================] - 0s 3ms/step - loss: 0.3728 - accuracy: 0.8877 - auc_3: 0.4585 - val_loss: 0.3971 - val_accuracy: 0.8729 - val_auc_3: 0.4273\n",
      "Epoch 3/10\n",
      "29/29 [==============================] - 0s 3ms/step - loss: 0.3663 - accuracy: 0.8877 - auc_3: 0.4370 - val_loss: 0.3932 - val_accuracy: 0.8729 - val_auc_3: 0.4188\n",
      "Epoch 4/10\n",
      "29/29 [==============================] - 0s 3ms/step - loss: 0.3613 - accuracy: 0.8877 - auc_3: 0.4428 - val_loss: 0.3902 - val_accuracy: 0.8729 - val_auc_3: 0.4197\n",
      "Epoch 5/10\n",
      "29/29 [==============================] - 0s 3ms/step - loss: 0.3599 - accuracy: 0.8877 - auc_3: 0.4478 - val_loss: 0.3882 - val_accuracy: 0.8729 - val_auc_3: 0.4303\n",
      "Epoch 6/10\n",
      "29/29 [==============================] - 0s 3ms/step - loss: 0.3584 - accuracy: 0.8877 - auc_3: 0.4568 - val_loss: 0.3863 - val_accuracy: 0.8729 - val_auc_3: 0.4507\n",
      "Epoch 7/10\n",
      "29/29 [==============================] - 0s 3ms/step - loss: 0.3571 - accuracy: 0.8877 - auc_3: 0.4684 - val_loss: 0.3847 - val_accuracy: 0.8729 - val_auc_3: 0.4723\n",
      "Epoch 8/10\n",
      "29/29 [==============================] - 0s 3ms/step - loss: 0.3560 - accuracy: 0.8877 - auc_3: 0.4803 - val_loss: 0.3832 - val_accuracy: 0.8729 - val_auc_3: 0.4911\n",
      "Epoch 9/10\n",
      "29/29 [==============================] - 0s 3ms/step - loss: 0.3550 - accuracy: 0.8877 - auc_3: 0.4961 - val_loss: 0.3819 - val_accuracy: 0.8729 - val_auc_3: 0.5139\n",
      "Epoch 10/10\n",
      "29/29 [==============================] - 0s 3ms/step - loss: 0.3541 - accuracy: 0.8877 - auc_3: 0.5085 - val_loss: 0.3807 - val_accuracy: 0.8729 - val_auc_3: 0.5290\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f06a4a35940>"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "single_model.fit(train_data,train_label,validation_data=(test_data,test_label),batch_size=128,epochs=10,shuffle=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summary\n",
    "The above two experiments simulate a typical vertical scene training problem. Alice and Bob have the same sample group, but each side has only a part of the features. If Alice only uses her own data to train the model, an accuracy of **0.872**, AUC **0.53** model can be obtained. However, if Bob's data are combined, a model with an accuracy of **0.893**  and AUC **0.883** can be obtained."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* This tutorial introduces what is split learning and how to do it in secretFlow  \n",
    "* It can be seen from the experimental data that split learning has significant advantages in expanding sample dimension and improving model effect through joint multi-party training\n",
    "* This tutorial uses plaintext aggregation to demonstrate, without considering the leakage problem of hidden layer. Secretflow provides AggLayer to avoid the leakage problem of hidden layer plaintext transmission through MPC,TEE,HE, and DP. If you are interested, please refer to relevant documents.\n",
    "* Next, you may want to try different data sets, you need to vertically shard the data first and then follow the flow of this tutorial\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
